<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
      <link rel="shortcut icon" href="../img/favicon.ico" />
    <title>shengteng - MKLorum</title>
    <link rel="stylesheet" href="../css/theme.css" />
    <link rel="stylesheet" href="../css/theme_extra.css" />
        <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github.min.css" />
    
      <script>
        // Current page data
        var mkdocs_page_name = "shengteng";
        var mkdocs_page_input_path = "shengteng.md";
        var mkdocs_page_url = null;
      </script>
    
    <!--[if lt IE 9]>
      <script src="../js/html5shiv.min.js"></script>
    <![endif]-->
      <script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/highlight.min.js"></script>
      <script>hljs.highlightAll();</script> 
</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav">
    <div class="wy-side-scroll">
      <div class="wy-side-nav-search">
          <a href=".." class="icon icon-home"> MKLorum
        </a><div role="search">
  <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get">
      <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" title="Type search term here" />
  </form>
</div>
      </div>

      <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <ul>
                <li class="toctree-l1"><a class="reference internal" href="..">Home</a>
                </li>
              </ul>
              <ul>
                <li class="toctree-l1"><a class="reference internal" href="../kunpeng/">kunpeng</a>
                </li>
              </ul>
              <ul class="current">
                <li class="toctree-l1 current"><a class="reference internal current" href="#">shengteng</a>
    <ul class="current">
    <li class="toctree-l2"><a class="reference internal" href="#_1">目录</a>
        <ul>
    <li class="toctree-l3"><a class="reference internal" href="#npu">一、部署到昇腾NPU</a>
        <ul>
    <li class="toctree-l4"><a class="reference internal" href="#1npu">1.查看NPU硬件信息</a>
    </li>
    <li class="toctree-l4"><a class="reference internal" href="#2npu">2.NPU环境下运行代码</a>
    </li>
        </ul>
    </li>
        </ul>
    </li>
    </ul>
                </li>
              </ul>
      </div>
    </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
      <nav class="wy-nav-top" role="navigation" aria-label="Mobile navigation menu">
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="..">MKLorum</a>
        
      </nav>
      <div class="wy-nav-content">
        <div class="rst-content"><div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href=".." class="icon icon-home" aria-label="Docs"></a></li>
      <li class="breadcrumb-item active">shengteng</li>
    <li class="wy-breadcrumbs-aside">
    </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
            <div class="section" itemprop="articleBody">
              
                <h1 id="olmo">OLMo適配昇騰云</h1>
<p>OLMo（Open Language Model）是由Allen AI（AI2，艾伦人工智能研究所）开发的一个完全开源开放的大型语言模型（LLM）框架，设计初衷是为了通过开放研究，促进学术界和研究人员共同研究语言模型的科学。OLMo框架提供了一系列的资源，包括数据、训练代码、模型权重以及评估工具，以便研究人员能够更深入地理解和改进语言模型。
<a href="https://github.com/allenai/OLMo">OLMo仓库地址</a></p>
<h2 id="_1">目录</h2>
<h3 id="npu">一、部署到昇腾NPU</h3>
<pre><code class="language-text">昇腾环境：
芯片类型：昇腾910B3
CANN版本：CANN 7.0.1.5
驱动版本：23.0.6
操作系统：Huawei Cloud EulerOS 2.0
</code></pre>
<h4 id="1npu">1.查看NPU硬件信息</h4>
<pre><code class="language-python">npu-smi info
</code></pre>
<p><img alt="image.png" src="https://bbs-img.huaweicloud.com/blogs/img/20241127/1732688508517690822.png" />
如果 Health 状态为 OK，说明 NPU 和 CANN 正常运行。</p>
<h4 id="2npu">2.NPU环境下运行代码</h4>
<p>代码拉取：</p>
<pre><code class="language-python">git clone https://github.com/allenai/OLMo.git

</code></pre>
<p>模型参数下载</p>
<pre><code class="language-python">pip install modelscope
modelscope download --model LLM-Research/OLMo-7B  --local_dir OLMo-7B
</code></pre>
<p>环境配置：创建python==3.10环境并激活</p>
<pre><code class="language-python">conda create -n py310 python=3.10
conda activate py310
</code></pre>
<p>进入OLMo文件路径下，</p>
<pre><code class="language-python">cd OLMo
pip install -e .[all]
</code></pre>
<p>新建run.py,</p>
<pre><code class="language-python">vim run.py
</code></pre>
<p>添加下边代码</p>
<pre><code class="language-python">import torch 
import torch_npu
from torch_npu.contrib import transfer_to_npu
from transformers import AutoModelForCausalLM, AutoTokenizer
olmo = AutoModelForCausalLM.from_pretrained(&quot;OLMo-7B&quot;)
tokenizer = AutoTokenizer.from_pretrained(&quot;OLMo-7B&quot;)
message = [&quot;Language modeling is &quot;]
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
# optional verifying cuda
inputs = {k: v.to('cuda') for k,v in inputs.items()}
olmo = olmo.to('cuda')
response = olmo.generate(**inputs, max_new_tokens=250, do_sample=True, top_k=50, top_p=0.95)
print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])

</code></pre>
<h5 id="21-torch_npu">2.1 下载torch_npu插件</h5>
<pre><code class="language-python">pip install torch_npu
</code></pre>
<p>运行代码：</p>
<pre><code class="language-python">python run.py
</code></pre>
<p>结果展示（略有差異）：</p>
<p><img alt="" src="https://bbs-img.huaweicloud.com/blogs/img/20241203/1733196937679484146.png" /></p>
<p>npu現存占用如下：</p>
<p><img alt="alt text" src="../picture/捕获.PNG" /></p>
              
            </div>
          </div><footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="Footer Navigation">
        <a href="../kunpeng/" class="btn btn-neutral float-left" title="kunpeng"><span class="icon icon-circle-arrow-left"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <!-- Copyright etc -->
  </div>

  Built with <a href="https://www.mkdocs.org/">MkDocs</a> using a <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
          
        </div>
      </div>

    </section>

  </div>

  <div class="rst-versions" role="note" aria-label="Versions">
  <span class="rst-current-version" data-toggle="rst-current-version">
    
    
      <span><a href="../kunpeng/" style="color: #fcfcfc">&laquo; Previous</a></span>
    
    
  </span>
</div>
    <script src="../js/jquery-3.6.0.min.js"></script>
    <script>var base_url = "..";</script>
    <script src="../js/theme_extra.js"></script>
    <script src="../js/theme.js"></script>
      <script src="../search/main.js"></script>
    <script>
        jQuery(function () {
            SphinxRtdTheme.Navigation.enable(true);
        });
    </script>

</body>
</html>
